Systems and methods for intra-operative image analysis

ABSTRACT

A system and method that acquire (i) at least a reference image including one of a preoperative image of a surgical site with skeletal and articulating bones and a contralateral image on an opposite side of the patient from the surgical site, and (ii) at least an intraoperative image of the site after an implant has been affixed to the articulating bone. The system generates at least one reference landmark point on at least one anatomical feature on the articulating bone in the reference image and at least one intraoperative landmark point on that anatomical feature in the intraoperative image. The reference and intraoperative images are compared, and differences between the orientation of the articulating bone in the two images are utilized to analyze at least one of offset and length differential.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.14/995,057, entitled “Systems and Methods for Intra-Operative ImageAnalysis,” filed Jan. 13, 2016, which is a continuation-in-partapplication of U.S. patent application Ser. No. 14/630,300, filed Feb.24, 2015, which claims priority to: (a) U.S. Provisional Application No.61/944,520, filed Feb. 25, 2014, (b) U.S. Provisional Application No.61/948,534, filed Mar. 5, 2014, (c) U.S. Provisional Application No.61/980,659, filed Apr. 17, 2014, (d) U.S. Provisional Application No.62/016,483, filed Jun. 24, 2014, (e) U.S. Provisional Application No.62/051,238, filed Sep. 16, 2014, (f) U.S. Provisional Application No.62/080,953, filed Nov. 17, 2014, and (g) U.S. Provisional ApplicationNo. 62/105,183, filed Jan. 19, 2015. The present application is alsorelated to U.S. patent application Ser. No. 14/974,225, filed Dec. 18,2015, by the present inventors, which issued as U.S. Pat. No. 10,433,914on Oct. 8, 2019. The entire contents of each of the above applicationsare incorporated herein by reference.

FIELD OF THE INVENTION

The invention relates to analysis of images of features within a patientand more particularly to accurately analyzing such images duringsurgery.

BACKGROUND OF THE INVENTION

Orthopaedic surgeons have the option of utilizing computer-assistednavigation systems to provide intraoperative surgical guidance. Forexample, computer navigation can provide data on functional parameterssuch as leg length and offset changes during hip arthroplasty. Thepurported benefits of computer navigation include reduction of outliersand adverse outcomes related to intraoperative positioning of surgicalhardware.

Despite obvious clinical benefit, these systems have had limitedadoption due to their expense, the learning curve and trainingrequirements for surgeons and, for some systems, the additionalprocedure and time associated with hardware insertion into the patient.Surgeons that do not use these systems are limited to traditionaltechniques that are generally based on visual analysis and surgeonexperience. However, these techniques are inconsistent, often leading tooutliers in functional parameters which may affect patient satisfactionand implant longevity.

Details of one such technique, specifically used in a minimally invasivehip arthroplasty technique referred to as the direct anterior approach,are mentioned in the description of a total hip arthroplasty surgery, byMatta et al. in “Single-incision Anterior Approach for Total hipArthroplasty on an Orthopaedic Table”, Clinical Ortho. And Related Res.441, pp. 115-124 (2005). The intra-operative technique described byMatta et al. is time-consuming and has a high risk of inaccuracy due todifferences in rotation, magnification and/or scaling of various images,because the technique relies upon acquiring a preoperative andintraoperative image that are scaled and positioned equivalently. Thetechnique also requires consistent patient positioning in thepreoperative and intraoperative images, including positioning of thefemur relative to the pelvis. Maintaining femoral position whileperforming hip arthroplasty can pose a significant and often unrealisticchallenge to a surgeon that is focused on performing a procedure. Thehigh risk of inaccurate interpretation using this technique has limitedits utility in guiding surgical decision making.

What appears to be a software implementation of this technique isdescribed by Penenberg et al. in U.S. Patent Publication No.2014/0378828, which is a continuation-in-part application of U.S. Pat.No. 8,831,324 by Penenberg. While the use of a computer system mayfacilitate some aspects of this technique, the underlying challenges tothe technique are consistent with the challenges to Matta's approach,and limit the system's potential utility.

The challenge of accounting for differences in femoral positioning,ever-present in existing non-invasive guidance technologies for hiparthroplasty, could be solved by developing a system and method thatcorrects for deviations between preoperative and intraoperative femoralpositioning.

It is therefore desirable to have a non-invasive system and method thatprovides intraoperative guidance and data by correcting for deviationsin femoral positioning between preoperative and intraoperative images.

SUMMARY OF THE INVENTION

An object of the present invention is to quantify restoration oforthopaedic functionality at a surgical site within a patient, evenduring a surgical procedure.

Another object of the present invention is to provide image analysis andfeedback information to enable better fracture reduction and/or optimalimplant selection during the surgery.

Yet another object of the present invention is to capture and preserve adigital record of patient results for data collection and qualityimprovements in surgical procedures.

A still further object of the present invention is to improve theoutcome of bone repositioning, fracture repair, and/or fixation within apatient.

This invention results from the realization that postoperative change inoffset and leg length can be accurately estimated during surgery byoverlaying or otherwise comparing preoperative and intraoperative imagesthat have been consistently scaled based on pelvic anatomy, generatingconsistent femoral landmarks in each image, and calculating the vectordifference between femoral landmarks after correcting for possibledifferences in femoral positioning between the two images relative tothe pelvis.

This invention features a system to analyze images at a surgical sitewithin a patient, the surgical site including at least one skeletal bonesuch as a pelvis and at least one articulating bone such as a femur thathas a longitudinal axis and articulates with the skeletal bone at ajoint. In one embodiment, the system includes an image capture modulecapable of acquiring (i) at least one reference image including one of apreoperative image of the surgical site and a contralateral image on anopposite side of the patient from the surgical site, and (ii) at leastan intraoperative image of the site after an implant has been affixed tothe articulating bone. A landmark identification module is capable ofreceiving the reference and intraoperative images and generates at leastone reference landmark point on at least one anatomical feature on thearticulating bone in the reference image and at least one intraoperativelandmark point on that anatomical feature in the intraoperative image.An image comparison module is capable of identifying (i) an estimationof at least the first center of rotation of the implant in at least oneof the reference image and the intraoperative image and (ii) thelongitudinal axis of the articulating bone in each of the referenceimage and intraoperative image. An analysis module is capable ofutilizing differences between the orientation of the articulating bonein the reference image relative to the orientation of the articulatingbone in the intraoperative image to analyze at least one of offset andlength differential.

In some embodiments, the first and second images are provided by theimage capture module to the landmark identification module in adigitized format. In certain embodiments, the analysis module calculatesa difference angle between the longitudinal axis of the femur in thereference image relative to the longitudinal axis of the femur in theintraoperative image and then estimates a corrected landmark point, suchas a corrected intraoperative landmark point, based on that differenceangle. In one embodiment, the analysis module estimates the correctedintraoperative landmark point by calculating a first radius between theestimated center of rotation and the intraoperative landmark and thenselecting the corrected intraoperative landmark point at a second radiusspaced at the difference angle from the first radius. In certainembodiments, the analysis module calculates length differential byestimating distance from the reference landmark point to the correctedintraoperative landmark point in a direction parallel to thelongitudinal axis of the femur in the reference image, and/or calculatesoffset by estimating distance from the reference landmark point to thecorrected intraoperative landmark in a direction perpendicular to thelongitudinal axis of the femur in the reference image.

In certain embodiments, at least one of the image comparison module, thelandmark identification module and the image comparison moduleidentifies at least one stationary point on the skeletal bone in each ofthe reference image and intraoperative image, and at least one of theimage comparison module, the landmark identification module and theimage comparison module aligns the reference image and intraoperativeimage according to at least the stationary point in each image. In oneembodiment, aligning includes overlaying one of the reference image andintraoperative image on the other of the reference image andintraoperative image.

In some embodiments, the reference image and the intraoperative imageare at least one of aligned and scaled relative to each other prior tothe analysis module analyzing offset and length differential. In oneembodiment, at least two stationary points are generated on the skeletalbone in the reference image to establish a reference stationary base andat least two stationary points are generated on the skeletal bone in theintraoperative image to establish an intraoperative stationary base, andat least one of the image comparison module, the landmark identificationmodule and the image comparison module utilizes the reference andintraoperative stationary bases to accomplish at least one of imagealignment and image scaling. In another embodiment, at least one of theimage comparison module, the landmark identification module and theimage comparison module provides at least relative scaling of one of thereference and intraoperative images to match the scaling of the other ofthe reference and intraoperative images.

This invention also features a system including a memory, a userinterface having a display capable of providing at least visual guidanceto a user of the system, and a processor, with the processor executing aprogram performing the steps of acquiring (i) at least one digitizedreference image including one of a preoperative image of a surgical sitewith skeletal and articulating bones and a contralateral image on anopposite side of the patient from the surgical site, and (ii) at leastone digitized intraoperative image of the site after an implant has beenaffixed to the articulating bone. The processor receives the referenceand intraoperative images and generates at least one reference landmarkpoint on at least one anatomical feature on the articulating bone in thereference image and at least one intraoperative landmark point on thatanatomical feature in the intraoperative image. The processor identifies(i) an estimation of at least the first center of rotation of theimplant in at least one of the reference image and the intraoperativeimage and (ii) the longitudinal axis of the articulating bone in each ofthe reference image and intraoperative image. One or more differencesbetween the orientation of the articulating bone in the reference imagerelative to the orientation of the articulating bone in theintraoperative image are utilized to analyze at least one of offset andlength differential.

This invention further features a method including acquiring (i) atleast one reference image including one of a preoperative image of asurgical site with skeletal and articulating bones and a contralateralimage on an opposite side of the patient from the surgical site, and(ii) at least one intraoperative image of the site after an implant hasbeen affixed to the articulating bone. The method further includesreceiving the reference and intraoperative images and generating atleast one reference landmark point on at least one anatomical feature onthe articulating bone in the reference image and at least oneintraoperative landmark point on that anatomical feature in theintraoperative image. The method includes identifying (i) an estimationof at least the first center of rotation of the implant in at least oneof the reference image and the intraoperative image and (ii) thelongitudinal axis of the articulating bone in each of the referenceimage and intraoperative image. One or more differences between theorientation of the articulating bone in the reference image relative tothe orientation of the articulating bone in the intraoperative image areutilized to analyze at least one of offset and length differential.

In some embodiments, aligning includes overlaying one of the referenceimage and intraoperative image on the other of the reference image andintraoperative image. In certain embodiments, the pelvis of the patientis selected as the skeletal bone and a femur is selected as thearticulating bone, and the skeletal component of the implant is anacetabular cup and the articulating bone component includes a femoralstem having a shoulder and pivotally connectable to the acetabular cupto establish the first center of rotation for the implant. The landmarkpoint on the articulating bone is identified to have a known locationrelative to the greater trochanter on the femur of the patient.

BRIEF DESCRIPTION OF THE DRAWINGS

In what follows, preferred embodiments of the invention are explained inmore detail with reference to the drawings, in which:

FIG. 1 is a schematic image of a frontal, X-ray-type view of a pelvicgirdle of a patient illustrating various known anatomical features;

FIG. 2 is a schematic diagram illustrating how multiple types of userinterfaces can be networked via a cloud-based system with data and/orsoftware located on a remote server;

FIG. 3 is a Flowchart G showing technique flow for both contralateraland ipsilateral analysis;

FIG. 4 is a Flowchart W of several functions performed for hip analysis;

FIG. 5 is an image of the right side of a patient's hip prior to anoperation and showing a marker placed on the greater trochanter as alandmark or reference point;

FIG. 6 is an image similar to FIG. 5 showing a reference line, drawn on(i) the pre-operative, ipsilateral femur or (ii) the contra-lateralfemur, to represent the longitudinal axis of the femur;

FIG. 7 is an image similar to FIG. 6 with a line drawn across the pelvicbone intersecting selected anatomical features;

FIG. 8 is a schematic screen view of two images, the left-hand imagerepresenting a pre-operative view similar to FIG. 6 and the right-handimage representing an intra-operative view with a circle placed aroundthe acetabular component of an implant to enable rescaling of thatimage;

FIG. 9 is a schematic screen view similar to FIG. 8 indicating markingof the greater trochanter of the right-hand, intra-operative image as afemoral landmark;

FIG. 10 is a schematic screen view similar to FIG. 9 with a referenceline drawn on the intra-operative femur in the right-hand view;

FIG. 11 is an image similar to FIGS. 7 and 10 with a line drawn acrossthe obturator foramen in both pre- and intra-operative views;

FIG. 12 is an overlay image showing the right-hand, intra-operativeimage of FIG. 11 superimposed and aligned with the left-hand,pre-operative image;

FIG. 13 is an image similar to FIG. 11 with points marking the lowestpoint on the ischial tuberosity and points marking the obturator foramenand top of the pubic symphysis in both pre- and intra-operative views;

FIG. 14 is an overlay image showing the right-hand, intra-operativeimage of FIG. 13 superimposed and aligned with the left-hand,pre-operative image utilizing triangular stable bases;

FIG. 15 is a schematic combined block diagram and flow chart of anidentification guidance module utilized according to aspects of thepresent invention;

FIG. 16 is an image of a trial implant in a hip with the acetabularcomponent transacted by a stationary base line and with two erroranalysis triangles;

FIG. 17 is a flowchart showing the use of an ‘Image Overlay’ techniqueto calculate a postoperative change in offset and leg length accordingto an aspect of the present invention;

FIG. 18 is a schematic diagram of an Image Analysis System according tothe present invention;

FIG. 19 is a schematic screen view of a preoperative image and anintraoperative image positioned side by side with digital annotationsmarking anatomic landmarks and stationary points on the images;

FIG. 20 is a schematic screen view of the preoperative image andintraoperative image of FIG. 19 overlaid according to pelvic anatomywith generated femoral landmark points and error analysis according toanother aspect of the present invention;

FIG. 21 is a schematic diagram showing generation of a correctedlandmark point and analysis of offset and length differential accordingto the present invention;

FIG. 22 is a schematic screen view of a preoperative image and anintraoperative image positioned side by side with a grid and digitalannotations to mark anatomic landmarks and other features on the imagesaccording to certain aspects of the present invention; and

FIG. 23 is a schematic view similar to FIG. 22 after the preoperativeimage has been aligned with the intraoperative image.

DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENTS

This invention may be accomplished by a system and/or method thatacquire (i) at least one reference image including one of a preoperativeimage of a surgical site with skeletal and articulating bones and acontralateral image on an opposite side of the patient from the surgicalsite, and (ii) at least one intraoperative image of the site after animplant has been affixed to the articulating bone. The reference andintraoperative images are received and at least one reference landmarkpoint is generated on at least one anatomical feature on thearticulating bone, such as on the greater trochanter of a femur, in thereference image and at least one intraoperative landmark point on thatanatomical feature in the intraoperative image. At least the firstcenter of rotation of the implant is estimated in at least one of thereference image and the intraoperative image, and the longitudinal axisof the articulating bone is identified in each of the reference imageand intraoperative image. One or more differences between theorientation of the articulating bone in the reference image relative tothe orientation of the articulating bone in the intraoperative image areutilized to analyze at least one of offset and length differential.

Broadly, some techniques according to the present invention, referred toby the present inventors as “Image Overlay”, place one image overanother image during analysis to generate a combined overlapped image.Previous approaches for the ‘Image Overlay’ technique made use of apelvic reference line having two or more points to scale and align apreoperative image and an intraoperative image. The pelvic referenceline having two or more points is also referred to as a “stationarybase” as defined in U.S. patent application Ser. No. 14/630,300 filed 24Feb. 2015, sometimes referred to herein as “parent application”, now USPublication No. 2015/0238271.

Alternative approaches for ‘Image Overlay’ technique according to thepresent invention obviate the need for the pelvic reference line orother stationary base. In some constructions, these alternatives insteadrely upon certain image acquisition techniques, certain imagemanipulation techniques, certain known imaging information, and/ordirect user manipulation to create consistent scale and alignmentbetween (i) at least one of a preoperative image and an invertedcontralateral image and (ii) an intraoperative image.

Additionally, any change in positioning of the femur in the two images,relative to the pelvis, would adversely affect calculations in previousapproaches of this technique. Maintaining femoral position whileperforming hip arthroplasty can pose a significant and often unrealisticchallenge to a surgeon that is focused on performing a surgicalprocedure. Various approaches for the ‘Image Overlay’ techniqueaccording to the present invention can correct for deviations in femoralpositioning between preoperative and intraoperative images bymathematically correcting for any deviation in femoral position in atleast one of the visual output and calculation output of offset and leglength. Presently preferred techniques, both with and without imageoverlay, are described in more detail below in relation to FIGS. 17-23.

In general, accurate analysis of two images of a patient is directlyrelated not only to how similar the two images are, but also howsimilarly the images are aligned with respect to scale and alignment,including rotation, and translation. Using conventional techniques, auser would have to manually adjust the images and/or retake multipleimages to achieve this goal, something that would be difficult to doreliably and accurately. Utilizing two or more points as a stationarybase according to the present invention in each image enables accurateanalysis of the two images. Furthermore, the present Image Overlaytechnique can analyze how “similar” these images are to give the userfeedback as to how accurate the results are, that is, to provide aconfidence interval. To obtain useful information, the images (the“intraop” intra-operative image and a “preop” pre-operative image, forexample) preferably are scaled similarly and rotated similarly, at leastrelative to each other.

For some constructions of image analysis according to the presentinvention, preferably at least one stationary base and at least oneanatomical landmark are selected, at least for scaling and alignment ofthe images. The term “stationary base”, also referred to herein as a“stable base”, means a collection of two or more points, which may bedepicted as a line or other geometric shape, drawn on each of two ormore images that includes at least one anatomical feature that ispresent in the two or more images of a region of a patient. For example,different images of a pelvic girdle PG of a patient, FIG. 1, typicallyshow one or both obturator foramen OF and a central pubic symphysis PS,which the present inventors have recognized as suitable reference pointsor features for use as part of a stationary base according to thepresent invention. Other useful anatomical features, especially to serveas landmarks utilized according to the present invention, includefemoral neck FN and lesser trochanter LT, shown on right femur F_(R),and femoral head FH and greater trochanter GT shown on left femur F_(L),for example. Femoral head FH engages the left acetabulum of the pelvicgirdle PG. Also shown in FIG. 1 are ischial tuberosities IT at thebottom of the ischium, a “tear drop” TD relating to a bony ridge alongthe floor of the acetabular fossa, and the anterior superior iliac spineASIS and the anterior inferior iliac spine AIIS of the ileum.

In general, a longer stationary base is preferred over a shorterstationary base, because the longer base, especially if it is a line,will contain more pixels in images thereof and will increase accuracy ofoverlays and scaling according to the present invention. However, thefurther the stationary base is from the area of anatomical interest, thegreater the risk of parallax-induced error. For example, if the area ofinterest is the hip joint, then the ideal stationary base will be nearthe hip. In some procedures involving hip surgery, for example, astationary base line begins at the pubic symphysis PS, touches orintersects at least a portion of an obturator foramen OF, and extends to(i) the “tear drop” TD, or (ii) the anterior interior iliac spine AIIS.Of course, only two points are needed to define a line, so only tworeliable anatomical features, or two locations on a single anatomicalfeature, are needed to establish a stationary base utilized according tothe present invention. More complex, non-linear stationary bases mayutilize additional identifiable points to establish such non-linearbases.

Additionally, at least one identifiable anatomic “landmark”, “stationarypoint” or “error point”, or a set of landmarks stationary points orerror points, is selected to be separate from the stationary base; theone or more landmarks, stationary points or error points are utilized incertain constructions to analyze the accuracy of the overlay process.This additional anatomic feature preferably is part of the stationaryanatomy being anatomically compared. For example, the inferior portionof the ischial tuberosity IT can be identified as an additionalstationary point or error point. This anatomic feature, in conjunctionwith the stationary base, will depict any differences or errors inpelvic anatomy or the overlay which will enable the physician tovalidate, or to have more confidence in, the output of the presentsystem. As generally utilized herein: (i) a “stationary point” refers toa point on a relatively stationary bone such as on the pelvis; (ii) a“landmark point” is located on an articulating bone such as a femur;(iii) an “error point” is preferably on pelvis and spaced from otherpoints; and (iv) a “fixed point” is located on an implant, such as theshoulder of a femoral stem prosthesis.

The term “trial hip prosthetic” is utilized herein to designate aninitial implant selected by a surgeon as a first medical device toinsert at the surgical site, which is either the right side or the leftside of a patient's hip in certain constructions. In some techniques,the trial prosthetic is selected based on initial digital templatingsimilar to the procedure described the parent application.

The term “digital representation” or “digital annotation” as utilizedherein includes a digital line having at least two points, e.g. a linerepresenting a longitudinal axis or a diameter of an implant or a bone,or a digital circle or other geometric shape which can be aligned withan implant or a bone intraoperatively and then placed in a correspondinglocation in a preoperative image, or visa versa.

FIGS. 2-16 herein correspond to FIGS. 4B, 7-16, 52-54 and 70,respectively, in the parent application. FIG. 2 herein is a schematicdiagram of system 141 according to the present invention illustratinghow multiple types of user interfaces in mobile computing devices 143,145, 147 and 149, as well as laptop 151 and personal computer 153, canbe networked via a cloud 109 with a remote server 155 connected throughweb services. Data and/or software typically are located on the server155 and/or storage media 157.

Software to accomplish the techniques described herein is located on asingle computing device in some constructions and, in otherconstructions such as system 141, FIG. 2, is distributed among a server155 and one or more user interface devices which are preferably portableor mobile. In some techniques a digitized X-ray image of the hip regionof a patient along a frontal or anterior-to-posterior viewing angle isutilized for a screen view on a display and, in other techniques, adigital photograph “secondary” image of a “primary” X-ray image of thehip region of a patient along a frontal or anterior-to-posterior viewingangle is utilized for the screen view. In one construction, the screenview is shown on a computer monitor and, in another construction, isshown on the screen or viewing region of a tablet or other mobilecomputing device.

Flowchart G, FIG. 3, shows technique flow for both contralateral andipsilateral analysis. This technique is commenced, step 340, and eithercontralateral or ipsilateral analysis is selected, step 342. Forcontralateral analysis, the contralateral hip image is captured, step344, and the image is flipped, step 346. For ipsilateral analysis, thepreoperative ipsilateral hip image is opened, step 348. For both typesof analysis, Flowchart W is applied, step 350.

Flowchart W, FIG. 4, after being activated by step 350, FIG. 3, guides auser to identify a femoral landmark such as the greater trochanter instep 370, FIG. 4, and then the femoral axis is identified, step 372,which corresponds to the longitudinal axis of the femur in that image.These steps are illustrated in FIGS. 5 and 6, below. A line is thendrawn across the bony pelvis, step 374, as shown in FIG. 7.

The technique proceeds to capturing an operative hip image, step 352,FIG. 3, and identifying an acetabular component, step 354, such as shownin FIG. 8 below. Acetabular components are also shown in and discussedrelative to FIGS. 9 and 10 below. The image is scaled by entering thesize of the acetabular component, step 356, and Flowchart W, FIG. 4, isthen applied to the operative hip, step 358. The operative andcomparative hip images are scaled by a stationary base generated byselecting at least two reference points on the bony pelvis, step 360,such as shown in FIG. 11. The scaled images are then overlaid in step362 using the bony pelvis points, such as the overlaid lines 386 and 412shown in FIG. 12. Differences in offset and leg length are calculated,step 364, and the technique is terminated, step 366.

One currently preferred implementation of the JointPoint IntraOp™Anterior system, which provides the basis for intraoperative analysis ofthe anterior approach to hip surgery, is illustrated in relation toFIGS. 9-22 in the parent application; FIGS. 9-16 are described herein asFIGS. 5-12. FIG. 5 herein is an image 376 of the right side of apatient's hip prior to an operation and showing a marker 378, bracketedby reference squares 377 and 379, placed by a user as guided by thesystem, or placed automatically via image recognition, on the greatertrochanter as a landmark or reference point. FIG. 6 is an image 376′similar to FIG. 5 showing a reference line 380, bracketed by referencesquares 381, 382, 383 and 384, drawn on (i) the pre-operative,ipsilateral femur or (ii) the contra-lateral femur, to represent thelongitudinal axis of the femur. FIG. 7 is an image 376″ similar to FIG.6 with a line 386, defined by two end-points, which is drawn across thepelvic bone intersecting selected anatomical features.

FIG. 8 is a schematic screen view of two images, the left-hand image376′ representing a pre-operative view similar to FIG. 6 and theright-hand image 390 representing an intra-operative view with a circle392 placed around the acetabular component 394 of an implant 398 toenable rescaling of that image. In some constructions, circle 392 isplaced by an image recognition program and then manually adjusted by auser as desired. Reference square 398 designates implant 398 to theuser. FIG. 9 is a schematic screen view similar to FIG. 8 indicatingmarking of the greater trochanter of the right-hand, intra-operativeimage 390′ as a femoral landmark 400, guided by reference squares 402and 404. FIG. 10 is a schematic screen view similar to FIG. 9 with areference line 406 drawn on the intra-operative femur in the right-handview 390″, guided by reference squares 407, 408, 409 and 410.

FIG. 11 is an image similar to FIGS. 7 and 10 with a line 386, 412 drawnacross the obturator foremen in both pre- and intra-operative views 376″and 390′″, respectively. Reference squares 413, 414, 415 and 416 guidethe user while drawing reference line 412.

FIG. 12 is an overlay image showing the right-hand, intra-operative,PostOp image 390′″ of FIG. 11 superimposed and aligned with theleft-hand, pre-operative PreOp image 376″. In this construction, softbutton icons for selectively changing PreOp image 376″ and/or PostOpimage 390′″ are provided at the lower left-hand portion of the screen.

Note that “PostOp” as utilized herein typically indicates post-insertionof a trial prosthesis during the surgical procedure, and is preferablyintra-operative. The PostOp image can also be taken and analysisconducted after a “final” prosthesis is implanted. “PreOp” designates animage preferably taken before any surgical incision is made at thesurgical site. In some situations, the image is taken at an earliertime, such as a prior visit to the medical facility and, in othersituations, especially in emergency rooms and other critical caresituations, the “PreOp” image is taken at the beginning of the surgicalprocedure. A ball marker BM, FIG. 5, is shown but not utilized foralignment because ball markers can move relative to the patient'sanatomy. Further PreOp and PostOp icons are provided in certain screenviews to adjust viewing features such as contrast and transparency.Preferably, at least one icon enables rotation in one construction and,in another construction, “swaps” the images so that the underlying imagebecomes the overlying image, as discussed in more detail below.

Additional icons and reference elements are provided in someconstructions, such as described in the parent application. One or moreof these “virtual” items can be removed or added to a screen view by auser as desired by highlighting, touching or clicking the “soft keys” or“soft buttons” represented by the icons. In certain embodiments, one ormore of the icons serves as a toggle to provide “on-off” activation orde-activation of that feature. Characters or other indicia can beutilized to designate image number and other identifying information.Other useful information can be shown such as Abduction Angle, OffsetChanges and Leg Length Changes, as discussed in more detail below.Optional user adjustment can be made by touching movement control icon527, FIG. 12, also referred to as a “rotation handle”.

In certain constructions, image recognition capabilities provide“automatic”, system-generated matching and alignment, with a reducedneed for user input. Currently utilized image recognition providesautomatic detection of selected items including: the spherical ballmarker frequently utilized in preoperative digital templating; theacetabular cup in digital templates and in trial prosthetics; and theCobb Angle line, also referred to as abduction angle.

In another construction, more than two points are generated for thestationary base for each image, such as illustrated in FIG. 13 for apreoperative image 1200 and a postoperative image 1201, and in FIG. 14for a combined overlay image 1298 of the preoperative image 1200 and thepostoperative image 1201 of FIG. 13. Similar locations on the pelvis ineach image are selected to generate the points utilized to establish astationary base for each image. In image 1200, for example, a firstpoint 1202 is generated on an upper corner of the obturator foramen orat the pelvic tear drop, a second point 1204 is generated at the top orsuperior portion of the pubic symphysis, and a third point 1206 isgenerated at the lowest or inferior point on the ischial tuberosity.Lines 1208, 1210 and 1212 are drawn connecting those points to generatea visible stationary base triangle 1216 on image 1200. Also shown is apoint 1214 on the greater trochanter. In postoperative image 1201, firstand second points 1203 and 1205 correspond with first and second points1202 and 1204 in image 1200. A third point 1207 is shown in image 1201between reference squares 1209 and 1211 in the process of a userselecting the lowest point on the ischial tuberosity to correspond withthird point 1206 in image 1200. The user is prompted by “Mark lowestpoint on Ischial Tuberosity” in the upper portion of image 1201. Alsoshown is a circle 1213 around the acetabular component and a point 1215on the greater trochanter.

Establishing at least three points is especially useful for determiningrotational differences between images. Overlay image 1298, FIG. 14,shows the three points 1202, 1204 and 1206 of preop image 1200, formingthe visible preop stationary base triangle 1216, which is positionedrelative to the corresponding three points 1203, 1205 and 1207 of postopimage 1201, forming a visible postop stationary base triangle 1311overlaid relative to triangle 1216 in FIG. 14. A ‘best fit overlay’ canbe created using these points by identifying the centroid of the polygoncreated by these point, and rotating the set of point relative to oneanother to minimize the summation of distance between each of therelated points. In this construction, scaling of the two images may beperformed by these same set of points or, alternatively, a separate setof two or more points may be utilized to scale the two images relativeto each other. Clicking on a PreOp soft-button icon 1300 and a PostOpicon 1301 enable a user to alter positioning of images 1200 and 1201,respectively, within image 1298 in a toggle-switch-type manner toselectively activate or de-activate manipulation of the selectedfeature. One or more points of a stationary base may be shared withpoints establishing a scaling line. Preferably, at least one landmark isselected that is spaced from the stationary base points to increaseaccuracy of overlaying and/or comparing images.

Also illustrated in FIG. 14 are “Offset and Leg Length Changes” with“Leg Length: −0.2 mm”, “Offset: 21.8 mm” and “Confidence Score: 8.1”. Aconfidence ratio that describes the quality of fit can be created bycomparing the overlay area of the two triangles relative to the size ofthe overall polygon formed by the two triangles, including thenon-overlapping areas of each triangle. Abduction angle and anteversioncalculations are described in the parent application in relation toFIGS. 55-59.

Alternative constructions may alternatively apply absolute scaling tothe preoperative and intraoperative images directly in each image, andwithout the need for a stationary base. For example, each image may bescaled by a ball marker or other scaling device, known magnificationratios of a radiographic device, or direct measurements of anatomicalpoints (such as a direct measurement, via callipers, of the extractedfemoral head, which can be used to scale the preoperative image).

Alternative constructions may also replace the ‘stationary base’ withvarious other techniques that could be used to scale and align thepreoperative and intraoperative images relative to one another. Oneexample of such a construction would involve overlaying two images anddisplaying them with some transparency so that they could both be viewedon top of one another. The user would then be prompted to rotate andchange their sizing, so that the pelvic anatomy in the two images wereoverlaid as closely as possible.

In some constructions, a guidance system is provided to adjust theviewing area of one image on a screen to track actions made by a user toanother image on the screen, such as to focus or zoom in on selectedlandmarks in each image. This feature is also referred to as anautomatic ‘centering’ function: as a user moves a cursor to ‘mark’ afeature on one image, such as placing a point for a landmark or astationary base on an intraoperative image, the other image on thescreen is centered by the system to focus on identical points ofinterest so that both images on the screen are focused on the sameanatomical site. FIG. 15 is a schematic combined block diagram and flowchart of an identification guidance module 1400 utilized in oneconstruction to assist a user to select landmarks when comparing a post-or intra-operative results image, box 1402, with a reference image, box1404. The module is initiated with a Start 1401 and terminates with anEnd 1418. When a visual landmark is added to a post-operative image, box1406, the module 1400 locates all landmarks “1” on the pre-operativereference image, box 1408, and calculates the visible area “v” withinthe pre-operative image in which to scale, such as by using

Equation 1:

v=[maxx(l)−minx(l),maxy(l)−miny(l)]  EQ. 1

The identical landmark on the pre-operative image is located and itscenter-point “c” is determined, box 1410. The identical landmark on thepre-operative image is highlighted in one construction to increase itsvisual distinctiveness, box 1414. The pre-operative image is centered,box 1410, and scaled, box 1412, such as by utilizing the followingEquations 2 and 3, respectively:

Center=c−(v)(0.5)  EQ. 2

Scale=i/v  EQ. 3

The user manipulates one or more visual landmarks in the results image,box 1416, as desired and/or as appropriate. In some constructions, theuser manually ends the guidance activities, box 1418 and, in otherconstructions, the system automatically discontinues the guidancealgorithm.

In certain constructions, image recognition capabilities provide“automatic”, system-generated matching and alignment, with a reducedneed for user input. Currently utilized image recognition providesautomatic detection of selected items including: the spherical ballmarker frequently utilized in preoperative digital templating; theacetabular cup in digital templates and in trial prosthetics; and theCobb Angle line, also referred to as abduction angle.

FIG. 16 is an overlay image 2000 of a preoperative hip image 2001 and anintraoperative hip image 2003 having a trial implant 2002 in a hip withthe acetabular component 2004 transacted by stationary base lines 2006and 2007 extending between a first point 2008 on the obturator foramenOF and a second point 2010 on the anterior inferior iliac spine AIIS ofthe ileum. Also shown are two error analysis triangles 2020 (solidlines) and 2030 (dashed lines). Circles 2022 and 2032 in thisconstruction represent a landmark point on the greater trochanter inimages 2001 and 2003, respectively. Image 2000 is a representation ofpreoperative and intraoperative hip images 2001 and 2003 overlaidaccording to stationary base lines 2006 and 2007, respectively. Threeidentical pelvic points 2024, 2026, 2028 and 2034, 2036, 2038 in images2001 and 2003, respectively, have been identified, with a system such assystem 200, FIGS. 4C-4F in the parent application, generating triangles2020 and 2030 for each image as represented by FIG. 16. The triangles2020 and 2030 can be visually compared to analyze the error in theanatomic area containing the stationary bases which, in this case, isthe pelvis. A numerical confidence score or other normalized numericerror analysis value may also be calculated and displayed in the systemby calculating the distance between points, comparing them to the lengthof the triangle vectors, and then normalizing the data, possibly using alog or other such nonlinear algorithm. The visual display and/ornumerical confidence score provides efficacy analysis in theconstruction. In other words, error analysis and correction is providedin some constructions for at least one image, such as providing aconfidence score or other normalized numeric error analysis, and/or avisual representation of at least one error value or error factor, suchas relative alignment of one or more geometric shapes, e.g. triangles,or symbols in two or more images.

In some constructions of the various alternative systems and techniquesaccording to the present invention, visual and/or audible userinstructions are sequentially generated by the system to guide the usersuch as “Draw line along Pubic Symphysis”. Guidance for surgeryutilizing other types of implants, and for other surgical procedures,including partial or total knee or shoulder replacements and footsurgery as well as wrist surgery, will occur to those skilled in the artafter reading this disclosure. Also, other types of medical imagingusing energy other than visible light, such as ultrasound, may beutilized according to the present invention instead of actual X-rays.Moreover, if a computer interface tool, such as a stylus or light pen,is provided to the user in a sterile condition, than the user can remainwithin a sterile field of surgery while operating a computing deviceprogrammed according to the present invention.

The term “vector” is utilised herein with the standard meaning of anEuclidean vector having an initial point or “origin” and a terminalpoint, representing magnitude and direction between the origin and theterminal point. The system then positions an acetabular componenttemplate or representative digital annotation, such as a digital line ordigital circle, in the preop image by replicating this vector.

Hip- and femur-related constructions of the present system and methodcalculate intraoperative changes in offset and leg length using areference image, also referred to as a “preop image”, and anintraoperative image, also referred to as a “postop image” or an“intraop image”. To accomplish this, one construction of the systemrequires two consistently scaled images that are overlaid and alignedaccording to the stationary anatomic region (such as the pelvis), thegeneration of at least one landmark point on the non-stationary,articulating anatomic region (such as the femur) in both images, amechanism to identify the difference in femoral angle of the femurrelative to the pelvis between the images, a mathematical correctionmodule that adjusts for differences in the articulating femur in eachimage relative to the stationary pelvis and, finally, a calculationmodule that uses this input to calculate intraoperative changes inoffset and leg length. As utilized herein, the term “femoral angle”refers to the orientation of the longitudinal axis of the femur relativeto the pelvis; a “difference in femoral angle” is described in moredetail below in relation to FIG. 21. The system may optionally includean error analysis module that identifies and analyses potential error inthe system.

As described in more detail below in relation to FIGS. 17-23, an ‘ImageOverlay’ process according to the present invention begins in someconstructions by acquiring (i) at least one of a preoperativeipsilateral or an inverted contralateral image (“preop image” or“reference image”), and (ii) an intraoperative image (“intraop image”).The system generates at least one landmark point on the non-stationaryfemur in both images (such as identification of a consistent point onthe greater trochanter in both images), generally performed with userguidance. Optionally, the system will generate at least one error pointon the pelvis in both images to provide error analysis. If the imageshave not been previously scaled and aligned, the system will scale andalign them using one of a plurality of techniques. One of the images isthen overlaid according to the pelvic anatomy in both images.

In some constructions, the system identifies points that can be used toanalyze possible error in the images relative to each other. The systemadditionally performs a series of steps to calculate any deviation inalignment of the non-stationary femur relative to the pelvic anatomybetween the preop and intraop images. The system then creates an overlayof the preop and intraop image, taking into consideration and correctingfor the effect of any difference in femoral angles between the twoimages as the system compares the relative position of the generatedfemoral landmark points. Finally, the system analyses the differencebetween the landmark points, including a correction for femoralalignment differences, and uses this data to calculate intraoperativechange in offset and leg length.

In one construction, the process begins in the flowchart OA in FIG. 17by acquiring, step 3000, either a selected preoperative ipsilateralimage, or a selected inverted contralateral image. Whichever image isselected is referred to herein as a “first, reference image” or “preopimage”. The process continues with acquisition of the intraop hip image,step 3002. Image acquisition in steps 3000 and 3002 is performed by theImage Capture module 3030, also referred to as an Image SelectionModule, of overlay analysis system 3028, FIG. 18. Acquisition of theseimages can be performed in a variety of ways, such as a directconnection to a c-arm fluoroscopy unit, file upload, or similartechniques. Implementations that operate on a mobile device such as aniPad, or other platforms that similarly integrate a camera device, mayalso acquire the images in steps 3000 and 3002 by prompting the user totake a picture of the images using the device camera. If an invertedcontralateral image is used as a ‘preop’ image, the contralateral imagemay be acquired and then inverted within the software, or otherwise itmay be flipped in another system and then input to image capture module3030. Screen view 3050, FIG. 19, shows preoperative image 3052 andintraoperative image 3070, referred to by labels 3053 and 3071 as“PreOp” and PostOp” images, respectively.

The method continues in step 3004, FIG. 17, with Landmark IdentificationModule 3034, FIG. 18, identifying at least one point on the femoralanatomy in both the preop and intraop images. Landmark IdentificationModule 3038 and Calculation Module 3040 can be considered as componentsof an Analysis Module 3037, shown in dashed lines. In a preferredconstruction, a point in each image will be placed on the greatertrochanter, a particularly useful landmark point because it is easilyidentifiable and because the anatomy is relatively insensitive todeviations in image acquisition. Alternatively, the point may be placedon the lesser trochanter or another identifiable femoral landmark.However, consistent point placement on the lesser trochanter is moresusceptible to error originating from deviations in image acquisitionangle based on its 3-dimensional anatomy. In various constructions, theuser is either prompted to identify the point on the femoral anatomy, orotherwise the system auto-identifies the point or set of points usingimage recognition or other technology and then allows the user to modifythe point placement.

FIG. 5, described above, is an image 376 of the right side of apatient's hip prior to an operation and showing a marker 378, bracketedby reference squares 377 and 379, placed by a user as guided by thesystem, or placed automatically via image recognition, on the greatertrochanter as a landmark or reference point, such as indicated inLandmark Identification Module 3034, FIG. 18. Reference squares 377 and379 enable the user to position the marker 378 on touch-screen devices,such as an iPad, without the user's fingers obscuring the position ofthe marker 378.

In a similar manner, reference landmark point 3054 and intraoperativelandmark point 3074, FIG. 19, are placed on the greater trochanter ofthe femur Fp in PreOp image 3052 and of femur Fi in PostOp image 3070,respectively. Also shown in PreOp image 3052 are a femoral axis line3055 and a pelvic reference line 3056, tear drop point 3056, pubicsymphysis point 3058, and ischial tuberosity point 3059.

Further shown in PostOp image 3070, FIG. 19, are acetabular cup AC andfemoral stem FS of an implant I, a femoral axis line 3075 and a pelvicreference line 3076, tear drop point 3076, pubic symphysis point 3078,and ischial tuberosity point 3079. A circle 3080 has been drawn aroundacetabular cup AC as described in more detail below.

In step 3006, FIG. 17, the Landmark Identification Module 3034, FIG. 18asks via User Interface UI, shown in phantom as box 3035, whether theuser wants to include error analysis in the system output. If yes,Module 3034 prompts the user, in Step 3008, to identify a set ofanatomic points on the stationary pelvis in both the preop and intraopimages. While a minimum of only one point is required to provide erroranalysis in the system, the system preferably generates at least threepoints on the pelvis, such as points 3057, 3058 and 3059 in PreOp image3052, FIG. 19, and points 3077, 3078 and 3079 in PostOp image 3070. Theuser positions each point on the pelvis in some constructions but, inpreferred constructions, automated algorithms of a system according tothe present invention initially place the points in appropriatepositions on the pelvic anatomy. If pelvic reference lines, as describedin more detail below, are used to align and scale the preop and intraopimages, the points selected for error analysis should be independent ofthe points used to create the pelvic reference lines. Ideal points willalso be identifiable, such as a discernible point on the pelvicteardrop, ischial tuberosity and pubic symphysis.

In Step 3010, the Landmark Identification Module 3034, FIG. 18,identifies the approximate femoral center of rotation in the intraopimage; this center of rotation information assists correction fordeviations in femoral positioning between the preop and intraop images.In a preferred construction, Landmark Construction Module 3034identifies this point by placing a digital circle so that it overlaysthe boundary of the acetabular component, as shown by digital circle 392in FIG. 9 and by circle 3080 in FIG. 19. The system then identifies themidpoint of the circle, which approximates the center of rotation of theacetabular component and functions as the intraoperative femoral centerof rotation.

Various constructions will accomplish step 3010 in different ways. In apreferred construction, the system may auto-detect the location of thedigital circle by using image recognition to auto-detect the acetabularcomponent in the intraoperative image, and then allow the user, via UserInterface UI, box 3035, to adjust the size and position of the digitalcircle using navigation handles connected to the circle, such asnavigation handle 527, FIG. 12, and by navigation handle 3099, FIG. 20.In another construction, the user estimates the approximate center ofrotation by drawing or positioning a circle around the femoral head inthe preoperative image, and utilizing the center of that circle as anestimate of the center of rotation.

As shown in FIG. 19, the PreOp image 3052 shows three error points 3057,3058 and 3059 positioned on the base of the pelvic teardrop, thesuperior point on the pubic symphysis, and the inferior point on theischial tuberosity, respectively. Similarly, points 3077, 3078 and 3079are positioned on corresponding points in PostOp image 3070. Thesecorresponding points will be used for error analysis in constructionsthat include error analysis as part of the system. Digital circle 3080has been positioned around the acetabular cup AC of implant I, with acenter-point represented by the crosshair 3081 that identifies themidpoint of the circle. This midpoint identifies the approximate femoralcenter of rotation after implant insertion.

In Step 3012, FIG. 17, the system begins the process of analysing thedifference in the femoral axis angles, relative to the pelvis, betweenthe preop and intraop images. In a preferred construction, the systemaccomplishes this by generating digital lines to identify thelongitudinal axis of the femurs in both images, such as femoral axislines 3055 and 3075, FIG. 19, and calculating any angle differencebetween them as described in more detail below in relation to FIG. 21.Landmark Identification Module 3034, FIG. 18 guides the user to generatea line that identifies the longitudinal axis of the femur in both thepreop and intraop images. First, the system generates a digital line inthe preop image to identify the femoral axis, and the system providesthe ability to adjust the line location so that it can identify theangle of the femur in the preop image. Then, the system generates adigital line in the intraop image to identify the femoral axis in theintraop image, again allowing for user adjustment. Preferredconstructions of this system will attempt to auto-identify the femoralaxis in this step using image recognition and known data, and place thedigital lines accordingly. The system then provides the functionalityfor the user to further manipulate these lines.

FIG. 6, described above, is an image 376′ similar to FIG. 5 showing areference line 380, bracketed by reference squares 381, 382, 383 and384, drawn on the preop image to represent the longitudinal axis of thefemur. Reference lines 381, 382, 383 and 384 can be manipulated toreposition the femoral axis line. FIG. 10, described above, is aschematic screen view with a reference line 406 drawn on theintra-operative femur in the right-hand view 390″, guided by referencesquares 407, 408, 409 and 410. Reference lines 407, 408, 409 and 410 canbe manipulated to reposition the femoral axis line. FIG. 19 again showsthe positioned digital lines 3055 and 3075, placed in Step 3012, FIG.17, that identify the femoral axis in the PreOp and PostOp images 3052and 3070.

In step 3014, FIG. 17, the Image Capture Module 3030, FIG. 18 determineswhether the preop and intraop images have been pre-scaled and alignedaccording to pelvic anatomy. Consistent scaling and alignment may bepreviously performed in this construction using a variety of approaches.For example, a software system residing on a digital fluoroscopy systemmay have been used to align and scale the images prior to imageacquisition by this system. Alternatively, the images may already bescaled and aligned because the surgeon took images with the patient andradiographic system in identical position with a known magnificationratio.

If the images have not been either scaled or aligned, the system canscale, or align, or scale and align the images in optional step 3016.Consistent scale and alignment in this step is accomplished by theoptional Image Scaling and Alignment Module 3032, FIG. 18, shown indashed lines, which may accomplish these operations in various ways.

One method to accomplish consistent scaling and alignment is by usingstationary bases (i.e. pelvic reference lines), along withidentification and scaling of the acetabular cup in the intraop image,as visually illustrated in FIG. 11. In this approach, a line is drawnconnecting two identical landmarks on the pelvis in both the preop andintraop images. Stationary base line 386 in FIG. 15 connects, in thepreop image, a point on the anterior superior iliac spine to theinferior point on the pubic symphysis. Stationary base line 412 in FIG.11 connects the identical two pelvic landmarks in the intraop image. Thesystem can use these two lines to rotate the images so that the overlaylines are aligned at the same angle relative to the software screen. Theimages can additionally be scaled, relative to one another, by scalingone image relative to another so that the pixel distances between thestationary base lines in the two images are equivalent. Finally,absolute scaling of the images can be achieved by scaling at least oneimage according to an object of known dimension.

FIG. 8 depicts the digital circle 392 that has been generated aroundacetabular component 394. The digital circle may be either generatedusing image recognition to identify the acetabular component, positionedby the user, or initially system-generated in an approximate locationand then positioned by the user. The size of this component is knownbecause the surgeon has placed it in the patient's femur. Therefore, theknown size of the component, such as “50” mm, can be entered into thebox following text “Size of Acetabular Component” located at the top ofthe intraop screen 390. The system uses this information to generateabsolute scaling in the intraop image. Additionally, the preop image canbe scaled in absolute measurements, according to this generated circle,once the preop image is scaled so that the pelvic reference lines inboth images are of equivalent length in pixels.

FIG. 19 depicts the pelvic reference lines 3056 and 3076 that have beengenerated on identical points on the preop and intraop images 3052 and3070 of the pelvis, allowing the system to align and scale the imagesaccording to the input. Alternative constructions may apply absolutescaling to other objects of known size in either the preop or intraopimage. For example, scaling can be applied according to the preop imageby drawing a digital line across diameter of the femoral head in thepreop image, and entering the size in absolute terms. This absolutemeasurement is known during surgery because the surgeon traditionallyextracts the femoral head and measures its size, using calipers, duringhip arthroplasty.

The output of the scaling and alignment performed in step 3016, FIG. 17,is used to generate an overlay in step 3018, and therefore may berepresented visually by depicting the updated scaling and alignmentvisually on the software screen, or otherwise may exclusively becalculated by the system to create the overlay in step 3018. In thisconstruction of Step 3018, the Image Comparison Module 3036, FIG. 18superimposes the preop and intraop images by aligning pelvic anatomy,with the images displayed with some transparency so that both can bevisualized in the overlay, such as illustrated in FIG. 20. In apreferred construction the overlaid images will contain the identifiedfemoral landmarks (generally placed on the greater trochanter) generatedin step 3008 so that location differences between the two points can bevisualized. The system will maintain the location of the generatedgreater trochanter points and the femoral axis lines, relative to thepreop and intraop images, as the images are manipulated to create theimage overlay.

The Image Comparison Module 3036 can align the images according topelvic anatomy in a variety of ways in this step. In a preferredconstruction, the system will have previously guided the user inidentifying at least two consistent points on the pelvic anatomy in bothimages. The Image Comparison Module 3036 then superimposes the images sothat the stationary base lines are positioned identically. In otherwords, the images are scaled, aligned and superimposed according to thestationary bases drawn across consistent points on the pelvis in eachimage. The Image Comparison Module will move and scale all digitalannotations in tandem with the underlying image so that they remainaffixed to the underlying image. This includes positioning of thefemoral and pelvic landmark annotations, the identified center ofrotation of the femur, pelvic reference lines, the femoral axis lines,and any other annotations used in various constructions.

Alternative constructions obviate the need for the use of the pelvicreference lines. In one alternative construction, the system uses imagerecognition technique to auto-identify the pelvic anatomy and overlaythe images based on the image recognition, then the user is presentedwith the option to manually manipulate the resulting overlay. In anotheralternative, the user will be guided to manually position the images sothat the pelvic anatomy matches. The system in this method will providethe user with the ability to manipulate both the position of each of theimages as well as adjust the magnification so that the pelvic anatomycan be superimposed on the overlay. Alternative systems will rely onhardware implementations and stationary cameras to obviate the need fora digital line, image recognition, or user manipulation whatsoever tocreate the overlay. In these instances, the external system may providea known magnification ratio and the consistent patient positioning thatwould be required to create the image overlay without the use of pelvicreference lines or similar technique.

Differences between the preop and intraop positioning of the femur,relative to the pelvis, creates a challenge in comparing the relativelocation of a femoral landmark such as a greater trochanter because achange in leg position alters the vector between the two femorallandmarks in the overlay. In Step 3020, FIG. 17, the Landmark CorrectionModule 3038, FIG. 18 calculates any existing difference between thepreop and intraop femoral axis angles. The terms “femoral angle” and“femoral axis angle” refer to the orientation of the longitudinal axisof the femur. If, for example, the preop and intraop femoral axis linesgenerated in step 3012 vary by eight degrees, the difference calculatedin step 3020 will be eight degrees.

In Step 3022, FIG. 17, Landmark Correction Module 3038, FIG. 18 usesdata gathered in previous steps to generate an additional “corrected” or“phantom” landmark point that accounts for differences in femoralposition between the preop and intraop images. A corrected landmarkpoint 3082 is shown in FIG. 20, positioned along circle 3083 fromintraoperative landmark point 3074′, which is similar to correctedlandmark point 3116, FIG. 21, along circle 3124 as described in moredetail below.

To generate the corrected landmark point, the module first calculatesangle_(femur), which is the angular difference between the longitudinalaxes of the femur in the preoperative and intraoperative images,respectively, also referred to as the preop and intraop femoral axislines in the overlay. This technique is shown schematically in FIG. 21for angle α, arrow 3108, between longitudinal axis lines 3104 (“L1”) and3106 (“L2”). The system incorporates this with the femoral or acetabularcenter of rotation 3102 (“R1”), (X_(origin), Y_(origin)) in the intraopimage, previously identified in step 3010, FIG. 17, and the greatertrochanter point 3110 (“p1”), (X_(troch), Y_(troch)) in the intraopimage. The system uses the following formulas to calculate the correctedlandmark “phantom” point 3116 (“p3”), (X_(phantom), Y_(phantom)) inEquations 4 and 5:

X _(phantom)=(X _(troch) −X _(origin))*cosine(anglef_(emur))−(Y _(troch)−Y _(origin))*sine(angle_(femur))+X _(origin)  EQ. 4:

Y _(phantom)=(X _(troch) −X _(origin))*sine(anglef_(emur))+(Y _(troch)−Y _(origin))*cosine(angle_(femur))−Y _(origin)  EQ. 5:

A vector “v”, line 3118, is extended from the preoperative landmarkpoint 3112 (“p2”) to corrected landmark point 3116. Right triangle“legs” 3120 and 3122 are utilized to estimate offset and leg length,respectively. Leg 3122 is generally parallel to preoperative femoralaxis 3104 in this construction. The Acetabular circle 3100 (“c1”)assists in locating center of rotation 3102. Also shown in FIG. 21 areradius lines 3130 and 3132 which are also separated by angle α, arrow3114.

As mentioned above, FIG. 20 is an “overlay” screen view 3050′ of theintraop image 3070, FIG. 19, superimposed as PostOp image 3070′ on thepreoperative image 3052 as PreOp image 3052′. The two stationary baselines 3056 and 3076 of FIG. 19 are aligned exactly one on top of theother, represented as a single stationary base line 3056′, 3076′. Firsterror correction triangle 3084 is shown connecting intraoperative errorpoint 3077′ on the pelvic teardrop, point 3078′ on the ischialtuberosity and point 3079′ on the pubic symphysis, and a similar errorcorrection triangle 3085 connects points 3057′, 3058′ and 3059′,representing points 3057, 3058 and 3059 of preoperative image 3052, FIG.19. Details window 3090 lists “Leg Length: −0.4 mm”, “Offset: −3.8 mm”and “Confidence Score: 5.4” as described in more detail below.

Finally, in Step 3018, FIG. 17, the Calculation Module 3040, FIG. 18,calculates the change in leg length and offset by analysing the vectorbetween the greater trochanter point in the preop image and thecalculated phantom point in the intraop image, such as illustrated inFIG. 21. To calculate leg length, the system calculates the distancebetween these two points along the femoral axis identified from thepreop image, as identified by line 3122 in FIG. 21. To calculate offset,the system calculates the distance between the two points along the axisthat is perpendicular to the femoral axis from the preop image, asidentified by line 3120. A specific example of these calculations isgiven in Details window 3090, FIG. 20.

The “Confidence Score” listed in box 3090 relates to the two errortriangles 3084 and 3085 as follows. The three points comprising eachtriangle enables the user to easily visualize any differences in pelvicanatomy in the overlay which may exist even after scaling and alignment.Although the stationary bases are completely matched one on top of theother, such as illustrated by single stationary base line 3056′, 3076′,the amount of deviation in the two error triangles 3084, 3085 can bevisually inspected to appreciate potential error in the system, such ascaused by one or more of parallax, differences in imaging vantage pointof the three-dimensional skeletal anatomy, and/or by point placementwithin the system.

As an additional, optional step to quantify the differences between theplacement of the two error triangles, the system provides a weighted“confidence score”, ranging from 0.0 to 10.0 in this construction. Inone implementation, the system finds the difference in an absolute scalebetween each of two corresponding points in the preop and postop imagesas overlaid. In some constructions, error in certain point pairs isassigned a weighting that is greater or lesser than for other errorpoint pairs. As one example, identifying a consistent point on theischial tuberosity may be difficult between images, so that particularpoint pair (labelled 3059′ and 3079′ in FIG. 20) can be weighted less,such as by “discounting” it by fifty percent. Finally, the weighted sumof numerical error among the error point pairs is converted to a singleconfidence score, such as “5.4” shown in display window 3090. Theweighting is not necessarily linear. Further, a cut-off value can beprovided beyond which the error is deemed to be too great to provideuseful analysis; in one construction, the system then recommends thatthe user obtain an alternative intraoperative image to compare with thepreoperative image, or with a contralateral image, to analyze accordingto the present invention.

Alternative constructions of this system and method will use differentmethods to determine the deviation between femoral angles in the preopand intraop images. For example, in one construction, the femoral anglecan be analysed by creating an image cut-out of one femur andsuperimposing it on top of the other at the original angle. The cut-outand underlying image may also be connected by the known femorallandmark, such as the greater trochanter, and be made to be immutable atthat single landmark point. Then, at least one of the system and usermay adjust the image cut-out so that the femoral bone precisely overlaysthe femoral bone in the superimposed image by pivoting about thatlandmark point. The system may accomplish this using image recognitionor other automated algorithm that identifies the femoral bone or relatedfemoral landmarks such as the greater trochanter landmark previouslyidentified. Alternatively, the user may match the femoral bones byadjusting the superimposed image of the femur so that it matches thefemur in the underlying image. The system may attempt to initially matchthe femoral bones and then provide the user the option to reposition thefemur to improve the position. Finally, the system will calculate thedeviation in angle between the two femurs by calculating the angle thatthe cut-out was adjusted, providing similar information

In yet another construction, reference (preop) and intraop images arecompared via a grid-type X-Y coordinate system without utilizing femoralangles, such as for preoperative images 3202, 3202′ and intraoperativeimages 3242, 3242′ in screen views 3200 and 3200′ illustrated in FIGS.22-23, respectively. The reference and intraoperative images are notactually digitally overlaid one on top of the other in thisconstruction; instead, preop image 3202, FIG. 22, is overlaid with, orotherwise associated with, a grid 3204 having a Y-axis 3205 and an Xaxis 3306 with units “100, 200, . . . 500” as shown, with the origin inthe upper left-hand corner of grid 3204. In a similar manner, intraopimage 3242 is associated with a grid 3244 having a Y-axis 3245 and an Xaxis 3346, preop image 3202′, FIG. 23, is associated with a grid 3204′having a Y-axis 3205′ and an X axis 3306, and intraop image 3242′ isassociated with a grid 3244′ having a Y-axis 3245′ and an X axis 3346′.

Preop image 3202, FIG. 22, includes femur Fp with landmark point 3208 onthe greater trochanter, and stationary base 3210 and error triangle 3212on the pelvis. Intraop image 3242 includes femur Fi with implant Ihaving femoral stem FS and acetabular cup AC. Intraoperative landmarkpoint 3248 has been placed on the greater trochanter. Stationary base3250 and error triangle 3253 have been placed on the pelvis.

Preop image 3202′, FIG. 23, includes femur Fp′ with landmark point 3208′on the greater trochanter, and stationary base 3210′ and error triangle3212′ on the pelvis. Intraop image 3242′ includes femur Fi′ with implantI′ having femoral stem FS' and acetabular cup AC′. Intraoperativelandmark point 3248′ is on the greater trochanter. Stationary base 3250′and error triangle 3253′ have been placed on the pelvis.

After a user activates a “Proceed To Analysis” icon 3260, FIG. 22, thesystem aligns preop image 3202′, FIG. 23, with intraop image 3242′. Inthis example, preop image 3202′ has been “tilted” or rotatedcounter-clockwise relative to the initial position of preop image 3202in FIG. 22 to represent alignment achieved using stationary base 3210′and 3250′. After both preop image and 3202′ and 3242′ have been alignedrelative to each other, then a difference in position of one of thelandmark points is determined, such as the shift of preop landmark point3208, FIG. 22 to the aligned position of preop landmark point 3208′,FIG. 23. In this example, intraoperative landmark point 3248′ is in thesame grid location as intraoperative landmark point 3248, FIG. 22. Avector can then be calculated from intraop landmark point 3248′ tocorrected point 3208′ using calculations similar to that described abovein relation to FIG. 21. In this construction, a “Details” window 3270graphically shows the change in position of initial preop landmark point3208 to corrected landmark point 3208′.

Other alternative constructions will change the order of various steps,including the generation of various digital landmarks. An additionalalternative construction will identify an estimated center of rotationin the preop image instead of the intraop image, using a similar digitalcircle placed around the femoral head, or similar technique to annotatethe estimate center of rotation.

Although specific features of the present invention are shown in somedrawings and not in others, this is for convenience only, as eachfeature may be combined with any or all of the other features inaccordance with the invention. While there have been shown, described,and pointed out fundamental novel features of the invention as appliedto one or more preferred embodiments thereof, it will be understood thatvarious omissions, substitutions, and changes in the form and details ofthe devices illustrated, and in their operation, may be made by thoseskilled in the art without departing from the spirit and scope of theinvention. For example, it is expressly intended that all combinationsof those elements and/or steps that perform substantially the samefunction, in substantially the same way, to achieve the same results bewithin the scope of the invention. Substitutions of elements from onedescribed embodiment to another are also fully intended andcontemplated.

It is also to be understood that the drawings are not necessarily drawnto scale, but that they are merely conceptual in nature. Otherembodiments will occur to those skilled in the art and are within thescope of the present disclosure.

What is claimed is:
 1. A processor-implemented method of estimating oneor more post-operative parameters during a surgical procedure, themethod comprising: registering a preoperative image and anintraoperative image based on an anatomical feature, wherein theanatomical feature is associated with an articulating bone capable ofarticulating relative to a first bone, wherein the preoperative imagecomprises a depiction of the first bone and the articulating bone, andthe intraoperative image comprises a depiction of at least one portionof the articulating bone, a surgical implant attached to thearticulating bone, and the first bone; determining, based on a firstcenter of rotation of the articulating bone in the registeredintraoperative image, an angle of rotation to align a longitudinal axisof the articulating bone in the registered preoperative image with acorresponding longitudinal axis of the articulating bone in theregistered intraoperative image; determining, based on the angle ofrotation, a corrected location of the anatomical feature in theregistered intraoperative image; estimating the one or morepost-operative parameters based on the corrected location of theanatomical feature in the registered intraoperative image; anddisplaying the one or more estimated post-operative parameters.
 2. Themethod of claim 1, wherein the surgical procedure is one of: a hiparthroplasty; or a knee replacement; or a shoulder replacement; or awrist replacement; or a foot replacement.
 3. The method of claim 2,wherein the surgical procedure is the hip arthroplasty, the first boneis a pelvic bone, the articulating bone is a femur, and the one or moreestimated post-operative parameters comprise at least one of a leglength, or an offset, or a combination thereof.
 4. The method of claim1, wherein the first center of rotation of the articulating bone in theregistered intraoperative image is determined based on one of: a secondcenter of rotation, wherein the second center of rotation is associatedwith the surgical implant in the registered intraoperative image, or athird center of rotation, wherein the third center of rotation isassociated with a digital representation of the surgical implant in theregistered intraoperative image.
 5. The method of claim 1, wherein thepreoperative image is either an ipsilateral image or a flippedcontralateral image.
 6. The method of claim 5, wherein the preoperativeimage is the flipped contralateral image and determining the firstcenter of rotation of the articulating bone in the registeredintraoperative image comprises: determining the first center of rotationof the articulating bone based on the flipped contralateral image. 7.The method of claim 1, further comprising: performing, prior to theestimation of the one or more post-operative parameters, at least oneof: scaling at least one of the registered preoperative image or theregistered intraoperative image, or aligning the registered preoperativeimage and the registered intraoperative image; or a combination thereof.8. The method of claim 7, wherein scaling at least one of thepreoperative image or the intraoperative image comprises one of: scalingthe registered preoperative image to a match an intraoperative imagescale; or scaling the registered intraoperative image to match apreoperative image scale; or rescaling the registered intraoperativeimage and the registered preoperative image to a consistent scale. 9.The method of claim 8, wherein the consistent scale to effect therescaling is determined based on one or more of: known parametersassociated with a radiographic device used to capture the preoperativeimage and the intraoperative image, or known dimensions of a markerappearing in the preoperative image and in the intraoperative image, ordirect anatomical measurements, or known dimensions of the surgicalimplant.
 10. The method of claim 7, wherein the scaling and the aligningare based on stationary base lines in the registered preoperative imageand corresponding stationary base lines in the registered intraoperativeimage, wherein each stationary base line connects feature points on thefirst bone in the registered preoperative image, and each correspondingstationary base line connects corresponding feature points on the firstbone in the registered intraoperative image.
 11. An apparatus toestimate one or more post-operative parameters during a surgicalprocedure, wherein the apparatus comprises: a memory to store apreoperative image and an intraoperative image, a display, and aprocessor coupled to the memory and the display, wherein the processoris configured to: register the preoperative image and the intraoperativeimage based on an anatomical feature, wherein the anatomical feature isassociated with an articulating bone capable of articulating relative toa first bone, wherein the preoperative image comprises a depiction ofthe first bone and the articulating bone, and the intraoperative imagecomprises a depiction of at least one portion of the articulating bone,a surgical implant attached to the articulating bone, and the firstbone; determine, based on a first center of rotation of the articulatingbone in the registered intraoperative image, an angle of rotation toalign a longitudinal axis of the articulating bone in the registeredpreoperative image with a corresponding longitudinal axis of thearticulating bone in the registered intraoperative image; determine,based on the angle of rotation, a corrected location of the anatomicalfeature in the registered intraoperative image; estimate the one or morepost-operative parameters based on the corrected location of theanatomical feature in the registered intraoperative image; and displaythe one or more estimated post-operative parameters on the display. 12.The apparatus of claim 11, wherein the surgical procedure is a hiparthroplasty, the first bone is a pelvic bone, the articulating bone isa femur, and the one or more estimated post-operative parameterscomprise at least one of a leg length, or an offset or a combinationthereof.
 13. The apparatus of claim 11, wherein the first center ofrotation of the articulating bone in the registered intraoperative imageis determined based on one of: a second center of rotation, wherein thesecond center of rotation is associated with the surgical implant in theregistered intraoperative image, or a third center of rotation, whereinthe third center of rotation is associated with a digital representationof the surgical implant in the registered intraoperative image.
 14. Theapparatus of claim 11, wherein the processor is configured to perform,prior to the estimation of the one or more post-operative parameters, atleast one of: scaling at least one of the registered preoperative imageor the registered intraoperative image, or aligning the registeredpreoperative image and the registered intraoperative image; or acombination thereof.
 15. The apparatus of claim 14, wherein to performscaling of at least one of the preoperative image or the intraoperativeimage, the processor is configured to: scale the registered preoperativeimage to a match an intraoperative image scale; or scale the registeredintraoperative image to match a preoperative image scale; or rescale theregistered intraoperative image and the registered preoperative image toa consistent scale.
 16. The apparatus of claim 15, wherein theconsistent scale to effect the rescaling is determined based on one ormore of: known parameters associated with a radiographic device used tocapture the preoperative image and the intraoperative image, or knowndimensions of a marker appearing in the preoperative image and in theintraoperative image, or direct anatomical measurements, or knowndimensions of the surgical implant.
 17. The apparatus of claim 14,wherein the scaling and the aligning are based on stationary base linesin the registered preoperative image and corresponding stationary baselines in the registered intraoperative image, wherein each stationarybase line connects feature points on the first bone in the registeredpreoperative image, and each corresponding stationary base line connectscorresponding feature points on the first bone in the registeredintraoperative image.
 18. A non-transitory computer-readable mediumcomprising instructions to configure a processor to: register apreoperative image and an intraoperative image based on an anatomicalfeature, wherein the anatomical feature is associated with anarticulating bone capable of articulating relative to a first bone,wherein the preoperative image comprises a depiction of the first boneand the articulating bone, and the intraoperative image comprises adepiction of at least one portion of the articulating bone, a surgicalimplant attached to the articulating bone, and the first bone;determining, based on a first center of rotation of the articulatingbone in the registered intraoperative image, an angle of rotation toalign a longitudinal axis of the articulating bone in the registeredpreoperative image with a corresponding longitudinal axis of thearticulating bone in the registered intraoperative image; determining,based on the angle of rotation, a corrected location of the anatomicalfeature in the registered intraoperative image; estimating one or morepost-operative parameters based on the corrected location of theanatomical feature in the registered intraoperative image; anddisplaying the one or more estimated post-operative parameters.
 19. Thecomputer-readable medium of claim 18, wherein the first bone is a pelvicbone, the articulating bone is a femur, and the one or more estimatedpost-operative parameters comprise at least one of a leg length, or anoffset or a combination thereof.
 20. The computer-readable medium ofclaim 18, wherein to estimate the one or more post-operative parameters,the instructions configure the processor to perform, prior to theestimation of the one or more post-operative parameters, at least oneof: scaling at least one of the registered preoperative image or theregistered intraoperative image, or aligning the registered preoperativeimage and the registered intraoperative image; or a combination thereof.